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Abstract 

We consider Markov processes, which describe e.g. queueing network processes, in a 
random environment which influences the network by determining random breakdown 
of nodes, and the necessity of repair thereafter. Starting from an explicit steady state 
distribution of product form available in the literature, we notice that this steady 
state distribution does not provide information about the correlation structure in time 
and space (over nodes). We study this correlation structure via one step correlations 
for the queueing-environment process. Although formulas for absolute values of these 
correlations are complicated, the differences of correlations of related networks are 
simple and have a nice structure. We therefore compare two networks in a random 
environment having the same invariant distribution, and focus on the time behaviour 
of the processes when in such a network the environment changes or the rules for 
traveling are perturbed. Evaluating the comparison formulas we compare spectral 
gaps and asymptotic variances of related processes. 


1 Introduction 

We consider stochastic networks of the Jacksonian type in a random environment. For a gen¬ 
eral introduction to Markov processes in random environments with applications to networks, 
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see [Zhu94] . |Eco05j . [ BM13j . These stochastic networks have recently found interest as a 
general model for queueing networks in connection with other areas of Operations Research, 
e.g. inventory theory and reliability theory. The interaction of network and environment is 
(i) in the queueing-inventory system that the service process decreases inventories and the 
inventory restricts serving customers due to limited stock at hand, and (ii) in the queueing- 
reliability system that external forces let servers break down which requires repair. 

We concentrate in the present paper on the second framework: External forces generate 
random breakdowns of servers in the network and the subsequent repair is also performed 
under random influences. We allow the environment to be of a rather general structure, 
which implies that nodes may break down in isolation or in groups, and that batch repair is 
possible as well. 

For this framework there is a product form extension of Jackson’s steady state result at hand, 
which provides in case of ergodicity the joint steady state distribution of the environment 
(represented by the set of broken down nodes) and the joint queue length vector in a product 
form: The environment status and the queue lengths seem to decouple asymptotically and 
in steady state (which is the essence of Jackson’s theorem in case of pure queueing systems). 
Clearly, this does not mean that environment and queue lengths are independent: The en¬ 
vironment is assumed here to be a Markov process for its own, but it strongly influences the 
service provided by the nodes and even the arrival streams there and, furthermore, the nodes 
interact as well - the interactions are carried by the traveling customers. These dependencies 
are not expressed by the one-dimensional (in time) marginal process distribution, which is 
a multidimensional (in space) product form distribution. In fact, very little is known about 
the dependence structure of the interacting processes. Therefore we study the correlation 
structure in time of the environment-queue length process via the one-step correlations (the 
Dirichlet forms of the associated Markov process), which in time as well as in space exhibit 
complex dependence behaviour. 

To be more precise, our interest is focused on the following scenarios: Compare two networks 
in a random environment which have the same invariant product form distribution, and so 
are in some sense variants of one another. Typical questions are: What happens to the time 
behaviour of a network when in such a network the rules for traveling (routing chains) are 
perturbed, or, when the environment changes? 

Our main results are comparison theorems and formulas which provide differences of one 
step correlations in related, resp. perturbed networks. Although the formulas for absolute 
values of the one step correlations are rather complicated, it turns out that differences of 
correlations of related networks are surprisingly simple and have a nice structure. As a con¬ 
sequence, whenever we have obtained quantities connected to one step correlations for some 
reference network as an anchor (possibly from simulations or numerical evaluations), we can 
perform easily explicit performance analysis, especially sensitivity analysis by varying, e.g., 
breakdown and repair probabilities or routing probabilities. 

The structure of the paper is as follows. In Section [2] we describe stochastic networks and the 
influence of the environment via Markovian breakdown and repair processes, which results 
in a non-Markovian structure of the queue size processes alone, and cite the steady state 
distribution for these networks. In Section [3] we derive the explicit formulas for the one 
step correlations in time for the joint environment-network process and show that for the 
interesting comparison problems these formulas simplify considerably. In Section [4] we show 
that our results allow to compare the spectral gaps and asymptotic variances of different 
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systems by evaluating our previous formulas suitably. Comparison results for spectral gaps 
allow to compare speed of convergence to stationarity for networks in L 2 norm. In Section 
Owe discuss relations to networks with finite buffers and extensions of our theorems to this 
area. Section E comprises the main technical proofs. 

Notation and conventions: 

For a set M we denote by 2 M = V(M) the set of all subsets of M. 

For sets A, B we write A C B for A which is a subset of B or equals B, and we write A C B 
for A which is a subset of B but does not equal B. 

5 xy is the Kronecker Delta, which is 1 iff x — y and 0 otherwise. 

Throughout, the node set of our graphs (networks) are denoted by J := {1,..., J}, and the 
’’extended node set” is J 0 := {0,1,..., J}, where ”0” refers to the external source and sink 
of the network. 

We denote the diagonal matrix with a vector £ on the diagonal and zero otherwise by diag (£). 
e-j is the standard j'-th base vector in N J if 1 <j< J and eo is the J-dimensional zero vector. 
Id is the identity matrix of appropriate dimension, defined by the context. 

For D C J and n = ( rij : j E J) E N J we write n D := ( rij : j E D) E and 
nj\ D := (rij : j E J \ D) E and will, as usual, identify n = {(rij : j E D), ( rij : j E 

J \ D)) = (rij : j E D,rij : j E J \ D). 

Similarly we use for N J -valued random variables with X t = X(t) — n— (rij : j E J) the self 
explaining abbreviations Xo(t) = (rij : j E D) E and X(t) = (Xo(t), Xj\ D (t)). 

For a probability space (E, £,tt) and functions /, g : (E, S) —> (R, B) we define the in¬ 
ner product of f,g with respect to 7T, whenever the following integral exists: (/,(?)* = 
hf{x)-g{x) 7T(dx). 

L 2 := L 2 (E, 7t) is the space of square integrable functions with respect to 7T, and ||/||jf = 

«/./>i) 1/2 . 

All random variables occurring are defined on a common underlying probability space (O, J 7 , P ). 


2 Stochastic networks in a random environment 

2.1 Stochastic networks 

A Jackson network [Jac57j consists of J nodes numbered 1,... ,J, where indistinguishable 
customers arrive, are served, possibly at several stations, and eventually depart from the 
network. The nodes are exponential single servers with state dependent service rates and 
with an infinite waiting room under first-come-first-served (FCFS) regime. If at node j 
there are rij > 0 customers present, either in service or waiting, then service is provided 
there at rate gj(rij ) > 0; we assume sup {fij(k) : j E {1 E Nj < oo and set 

jiij(O) := 0. All customers follow the same rules. We shall need later on a slight extension of 
the standard Jackson network models. This is described in terms of an irreducible stochastic 
routing matrix 

R — ( 2 - 1 ) 

where the artificial ’’node 0” represents the source and the sink of all customers. Strict 
inequality may hold for r 00 > 0, which means that some arriving customers may be rejected. 
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Customer arrive in a Poisson stream of intensity A > 0 which is split (independently) ac¬ 
cording to the first row r 0 := (r 0i : i — 0,1,..., J) of R. Then at nodes j — 1, 2,..., J we 
observe independent Poisson- Xj arrival streams with A j = Xr 0 j, while a portion Ao = Ar 0 o of 
the arriving customers is rejected (lost). 

Routing is Markovian, a customer departing from node i immediately proceeds to node j 
with probability r i3 > 0, and departs from the network with probability r j0 . 

Then the traffic equations for the admitted customers 

j 

Vj = + , J, ( 2 - 2 ) 

i —1 

have a unique solution (■ r\ 3 : j = 1J). Note, that (12.2[) only counts for the admitted 
customers because of Xj = A • r 0 j,j = 1,..., J, and Ai + ... + Xj = A(1 — r 00 ). In case 
of r 0 o = 0, R is the so-called extended routing matrix of standard Jackson networks, see 
[Ml [(3.2)]. 

Let X = ( X t : t > 0) denote the vector process recording the joint queue lengths in the 
network at time t. X t = (Xi(t),... ,Xj(t )) G N J reads: at time t there are X 3 (t) customers 
present at node j , either in service or waiting. The assumptions put on the system imply that 
X is a strong Markov process on state space N J with generator Q x = (g x (n, m) : m, n G N J ) 
which is given for g : N J —?• R by 

j J 

{Q X g)(n) = X o{a{n + ej) - g(n)) + ^(%)r j0 (^(n - e 3 ) - g(n)) 

j= i j= i 

j j 

+ ^ J Hj{n j )'^ j r ji (g{n-e j + e i ) - g{n)) (2.3) 

j =i *=i 


Q x is a bounded operator because of inf q x (n,n ) > —oo. We assume throughout that 
X is ergo die. 

For an ergodic network process X Jackson’s theorem [Jac57] states that the unique steady- 
state and limiting distribution 7r on N J is with normalizing constants C(j ) for marginal 
distributions of X 


7r(n) = 7r(ni, 



n 


Vj 


L = i Vj(k) 


n = (m,..., nj) G N J 


(2.4) 


2.2 Breakdown-repair processes 

We are interested in stochastic networks, where nodes due to external environment influences 
can breakdown and are repaired periodically. A common situation is that the breakdown- 
repair process is Markov of its own, and the network reacts on the perturbations driven 
by the random environment. To describe these Markovian processes we consider a set of J 
stations or devices (nodes) numbered 1,... ,J. Stations are unreliable, break down randomly 
and are repaired thereafter. Repair time is random as well. We assume that the availability 
status of the system can be described by a homogeneous Markov process 
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Y = (Y(t) :t> 0), Y(t) : (O ,F,P) -+ (2 J , T(2 J )). 

Y(t) = D indicates that at time t > 0 the stations in D C J are down and under repair, 
while stations in J\D C J are functioning (’’are up”). The transition rates (breakdown and 
repair intensities) of Y are 


Definition 2.1 Take any pair of functions A : 2 J —y [0, oo) and B : 2 J —» [0, oo), subject 
to A(fjf) = 1 and B(fjf) = 1 and for D,I,H C J (we set 0/0 = 0 and 1/0 = oo) subject to 

^ \ < oo Vhc/C J and —;—- <oo V H C D C J . 

A(D) - B(H) 

With these functions define breakdown and repair rates as follows: 

for breakdowns of nodes in non-empty set I \ D if nodes in D are already down, and 

qY(D ' H) = Wi)’ HcD - J - 

for finishing repair of nodes in non-empty set D\H if nodes in D are under repair. For all 
other pairs G, H C J,G ^ H, we set q Y (G, H) = 0, and for all D C J we set q Y {D, D ) = 
— ^2hcj,h^d H) ■ 


The generator Q Y = (g Y (A", L) : K, L C J) of Y is defined for real functions / : 2 J — » M, by 

(Q y /)(D) = Ff-U(H) - f(D)) + 4m(fW - f(°)) (2.5) 

By evaluation of the standard local balance equations it can be seen that Y is reversible 
with respect to the probability measure (with normalization constant C ~ l ) 

* := (*(£>) := C- 1 Ak), DE2'). (2.6) 


2.3 Rerouting 

The network process and the breakdown-repair process (availability process) interact and 
we have to fix rules for the interaction regime. The general rule is: 

(1) Whenever a station is broken down and under repair, service is interrupted and the 
customers present there are frozen, while new customers are not admitted to this station. 

(2) Therefore we have to define a new routing mechanism. Examples of how to do this to 
obtain explicit steady states can be found in |SD03j [Sections 5, 6]. We describe an abstract 
’’rerouting scheme”, which encompasses the three schemes described there. 
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Assumption 2.2 Rerouting schemes in open networks. Consider a Jackson net¬ 
work with routing matrix (12.ip (with (r 00 > 0)^ and traffic equations for the admitted cus¬ 
tomers O where Xj = Xr 0 j. 

When nodes in D C J are down, routing is restricted to Jo \ D and determined by some 
routing matrix 

= Rlww,' < 2 '7) 

The associated traffic equations for the admitted customers similar to (12. 2 p are 

vf = X f + j £ J\D, with Xf : = X • r£, j £ J\D, (2.8) 

i£j\D 

and are assumed to be solved by 


vf=Vji j£J\D, (2.9) 

where the r/j are from the solution of (12.2H . We set i? 0 := R and, if necessary, := rjj. 

For the rerouting scheme with nodes in set D broken down X(f := A -r(f 0 is the new rejection 
rate. 

Lemma 2.3 If (r/j 5 , j £ J\D) solves the traffic equations (12.8p for the admitted customers, 
when nodes in D are broken down and rerouting is according to Assumption 12.21 then with 
q[f := X the vector (qf ,j £ Jq\D) solves the equation x = x ■ R D . 

Proof. (12.8p can be written as q^ = A • + Y2ieJ\ D vF r ij^ J ^ J \ D. Summing over 

j £ J \ D yields A(1 — r[[ 0 ) = Y^neJ\o r lP r M > which is the missing equation of x — x ■ R D , 
with the recjuired solution inserted. □ 


Remark 2.4 (i) For technical simplifications we introduce the normalized probability vector 
( D associated with values qj ) : = Vj/(J2i =0 Vk),j £ Jo\D- Similarly we dehne f = £ 0 

associated with the qj = pj. 

(ii) Prescribing rerouting by (j2.8j) is not constructive, but is not necessary for our main 
applications. A detailed description of rerouting schemes which fulfill Assumption 12.21 is 
given in |Sau06j [Section 2], 

(iii) When considering rerouting schemes which are used in the literature it may happen that 
the rerouting chain on certain subsets J\D is not irreducible, for details see [SauOfi j [Proof of 
Theorem 1.2.29]. This makes the computations more involved, but leads to the same results 
as those we shall present below. 


Example 2.5 The most common rerouting schemes found in the literature which lead to ex¬ 
plicit stationary distributions of the network processes are (for more details see ISDOS f [Sections 
5, 6] and IDS 13f [Section 2.3]) 

(i) RS-RD with reversible routing. This applies only for reversible R. When a cus¬ 
tomer tries to visit a down node he is rejected, stays at his node for another service and tries 
again with newly selected destination. 


6 
















(ii) Stalling. Whenever any node breaks down all services and arrivals are interrupted 
and resumed only when all nodes are up again. 

(in) Skipping. Whenever a customer wants to visit a broken down node he is not allowed 
to settle down there and has to jump forward according to R until he reaches an up-node or 
leaves the network. 

2.4 Networks with breakdown and repair: Product formula 

A Markovian state process for an unreliable Jackson network requires that the state space 
N J of the Jackson network process X is supplemented by a coordinate Y which indicates 
the set of broken down stations. Operating on these states we define a Markov process 
Z = (Y, X) describing the degradable network with state space E = 2 J x N J . States are 
n = ( D,n ) = (D,ni,n 2 , ■ ■ ■ ,nj) e E, where the first coordinate in n we call the availability 
coordinate. The interpretation is: D is the set of servers in down status. At node i G D 
there are n* customers waiting for server being repaired. We denote by S 2 E . 

Definition 2.6 The unreliable Jackson NETWORK process is the Markov process 
Z = ( Z(t),t > 0) defined by the infinitesimal generator (transition intensity matrix) Q z = 

( q z (n, n!) : n, n' £ E ) via 

(Q Z f)(D, ni ,n 2 ,...,nj) = Y Xr oj(f( D ,H + e j) ~ f( n )) + Y Uj( n j) r jo(f( D ,n- e j) ~ f( n )) 

j&J\D jeJ\D 

+ Y ViM r ?iU( D ,n~ e 3 + e i) -/O)) (2T0) 

jeJ\D i£j\D 

+ X Hirt/Ws) - /(»)) + X - /(»))• 

HCD ^ ' IdD ^ ' 

Theorem 2.7 Product form for Jackson networks with breakdown and re¬ 
pair. \SD0S j . 

lSau()6\/ [Theorem 2.4-1] Under the Assumption ^. 21 if Z = (Y, X) is ergodic then the steady 
state is with i r from (12.4j) and k from (12.6jl of the product form: For (D, n ±,..., nj) G E 

t t(D, ni, n 2 ,.. •, nj) = vr(ZJ) • n(ni,n 2 , ...,nj) = | ' Cj l -^L- j . (2.11) 

Here r] = (rji,, r\j ) is the solution of the traffic equation (12. 2\ for admitted customers when 
all nodes are up, Cj is the normalization constant for the local queue length process at node 
j. We denote C = n/=i Cj • 

(12.11(1 is proved in (i SD03l| for more general breakdown and repair schemes: Breakdown and 
repair rates may depend on the load (queue lengths) of nodes. The question whether in this 
framework results similar to those in the following sections can be derived is still open and 
part of our ongoing research. 

Theorem 12.71 is not covered by the results for networks in a random environment in |Zhu94j 
and [ Eco05j . In both papers it assumed that under different environment states the ratio 
’’local arrival rate/local service rate” is independent of the environment status. This is 
obviously not the case in our systems. 
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3 One step correlation 


Recall Xj = A • r 0j , j = 1,..., J, and Ai + ... + Xj = A(1 — r 0 o) and that we therefore consider 
only admitted customers even if all nodes are up. We will not mention this further. For 
the network process Z with generator Q z and stationary distribution 7 f consider one step 
correlation expressions 

(f,Q z gh- (3.1) 

If / = g, then (13.11) is (the negative of) a quadratic form, because — Q z is positive definite. 
(13.11) occurs in the definition of Cheeger’s constant because division of (13.ip with f = g 
by (/? f)n yields Rayleigh quotients. It also occurs in the definition of the corresponding 
Dirichlet form. This is helpful to bound the second largest eigenvalue of Q z and to prove the 
Poincare inequality for the corresponding Markovian process, see e.g., . (’lid) 1 . Furthermore, 
(13.11) can be utilized to determine the asymptotic variance of costs or performance measures 
associated with Markovian network processes and to compare the asymptotic variances of 
two such related processes. It is possible to compare the correlations for Z with that of the 
related process Z' with the same stationary distribution 7 f, using (/, Q z g)jr—(f, Q z g ) n , which 
will be given explicitly in Section [3721 Because we are dealing with processes having bounded 
generators, properties connected with (13.11) can be turned into properties of (/, Id+eQ z g)^ = 
E ^( f ( Z 0 ) g ( Z r )) where s > 0 is sufficiently small such that Id + eQ z is a stochastic matrix 
and r is exponentially distributed. This enables one to directly apply discrete time methods 
to characterize properties of continuous time processes. 

3.1 Correlation formulas 

Due to the product form steady state distribution of Z the one step correlation (/, Q z g )* 
splits immediately into two terms having an intuitive interpretation: The sum of weighted 
one step conditional correlations 

(i) of the environment process Y, and (ii) of network processes, which for a fixed time point 
seem to behave conditionally independent of the environment. 

As will be seen, it is illuminating to define for all DC J the generators Q Ed Q f ’’synthetic 
subnetworks” on node set J\D with overall arrival rate A, service rates from Definition 12.61 
and routing matrix R°. 

The proofs of Propositions 13.11 and 13.21 are postponed to Section [HI 

Proposition 3.1 Splitting formula. For unreliable Jackson network processes Z = 
(Y, X) the one step correlations splits as follows 


(f, Q z 9)* = Y 71 ^ { Y *( D )f( D > a) (<5 y d(-, a)) {d) > + Y ^( D ) Y 


kd{H d ) 


igN KDQJ 


D ^ J n D GN 


PI 


Y 7r J'\n(Aj\D)/( £) > (Md,!U\d )) (<2 XjV3 fKA (Hd> (-)j\r»))) (z*j\£>) 

N ^-P! 














The next correlation formula will yield remarkable simplifications when used for differences. 


Proposition 3.2 One-step correlation formula. For unreliable Jackson network 
processes Z = (Y, X) with ( D the probability solution of x D = x D ■ R D (when nodes in 
D are down) holds 

{/. Q z g)i = X X 

DC{1,...,J} r£N J 

ReN J 0 [jeJo\DieJo\D 

- X X ^(^5 n.) f (tt' U-)fji c■ zz) i ^ ^ q (-^j -^0 ^ ^ q (D, /) + A + ^ ^ ) 

DC{1,...,J} pjJ (fiCO Ofl j£J\D 


n 


Y C J Y (D, H)f(D,n)g(H, n) + £ g Y (A /)/(£>, n 


.HcD 


IDD 


3.2 Comparison of one step correlations 

The following formulas for differences of one step correlations will give additional insight into 
various properties of networks, for example to speed of convergence or asymptotic variance. 
They display how, e.g., the routing and the breakdown and repair affects correlations in 
networks. 


Theorem 3.3 Changes of routing behaviour of the customers. Suppose Z = 
(Y, X) is an ergodic unreliable Jackson network process with a routing matrix R and TJ = 
(Y', X') is another Jackson network process having the same arrival and service intensities 
and failure-repair rates but with routing matrix R' = [rb]ij = suc h that the solutions 
of the traffic equation derived from R and for R' coincide (denoted byrj). Assume that both 
networks follow some rerouting mechanism for which the Assumption 12.21 holds. Then for 
arbitrary real functions f,g G L 2 


(f,Q z g)ir - ( f,Q z 'g)n = E, 


A (tr(W^(Y„ X,) ■ diagdp ■ (R Y ‘ - R Y ))) 

-SO v 7 


where ( D is the probability solution of x D = x D ■ R D , tr(A ) denotes trace of A, and 

W 9j (D,n) = [^(T»,n + e i )/(T),u + e J )] ij . e j o ^. 


Proof. Because the external arrival streams are the same, and the traffic equations have the 
same solution 77 , and the rerouting mechanisms have property (12.91) . for any availability status 
D the rerouting schemes on J\D have the same solution of the traffic equation. It follows 
from Lemma 12.31 that for all D the probability solution of the equations x D = x D ■ R D and 
X D — X D . R’ d are in both systems the same. Because of q Y = q Y we have from Proposition 
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13.21 the reduction 


nt 


(f,Q z 9),-(f,Q z ’g),= E c-#(£>) E ML.m 

DC(1....,J} „ 6 [sj|Dlt6Dl.l V'M ' 


Vi 


na 


—J\D 


e nn 

e e j\ D 1=1 


Vi 

Vi{i) 


A 


£ 0 

A 


D 


ID 


Y1 Y1 ^f r fif( D ^+e j )g{D,n + e i ) 

k j£Jo\D i£]o\D 

tfr'fif&’Tk + eJgiD^ + ei) 

K j&Jo\D i£Jo\D 


We interpret in the last two lines for fixed D and n D and i, j G J 0 \ D the expressions 

f(D, n + ef) =: f(D , (n D , nj\ D + e,)) and g(D, n + ef) =: g(D, (n D , nj\ D + e*)) 

as functions of nj\ D only, and see that the resulting expressions have exactly the structure of 
the functions dealt with in Proposition 4.1 of |DS08j . After renormalization of the densities 

n^e./\D ni^i (iEpj)’ w hich in fact results in conditioning on {Y(t) = D,Xo(t) = n D }, we 
obtain (/, Q z g )* - (/, Q z 'g)* 


E E n{y(t) = D,x D (t) = n D }) 

DC{l,...,J} lU)€N IO| 

A (tr(W’ J (D, (n D ,Xj^ D )) ■ dmg( n ■ ( R° - |{V(t) = D,X D (t) = n D } 

.SO v 7 


■E; 


and deconditioning eventually finishes the proof. □ 


Theorem 3.4 Changes of breakdown and repair mechanisms. Suppose Z = (Y, X) 
is an ergodic unreliable Jackson network process with a routing matrix R = [rij]ij= o,i,...,j and 
Z' = (Y', X') is another Jackson network process having the same arrival and service inten¬ 
sities, and with the same routing regime, described by R and rerouting fulfilling Assumption 

B 

The breakdown-repair process for Z is given in Definition \2.1\ and is for TJ defined similarly 
via A ', B' : 2 J —y [0, oo), subject to the restrictions indicated there. Then the breakdown and 
repair rates for Y' are: 

qY(D ’ I) = jr~D)' Dc/ - J ’ and HCDCI. 


The processes Y and Y' are Markov with generators Q Y = ( q Y (K , L) : K, L C J) of Y and 
Q Y ' = ( q Y '{K,L) : K,L C J) of Y' as defined in (I2.5[) for Y and similar for Y'. 

Assume that the stationary distributions of Y and Y' are identical, denoted by 




B(D) 


B'{D) ’ 


D e 2- 
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Then for arbitrary real functions /, g : E —y K holds 

(f,Q z ah - (f,Q z 'g)» = K \{f(°,x t ),Q Y g (; x,)(°)>* - {f(o,x t ),Q Y ' g (;X t )(o)) t 


= E r T 


(/(o,x,),(Q Y -Q Y M-,x t )(o)). 


Proof. Interchanging summations, regrouping terms, and exploiting the product form struc¬ 
ture of the state distributions (which are identical for Z = (Y, X) and ZI = (Y',X')) in 
(13.21) we obtain 


( f,Q z g)n = 


Y Y ^{D) <J Y 

dc{i,..,j} Ihcd idd 

~ Y 7r (^) E K( D )f( D ,n)g(D,n) j Y Q Y ( D > H ) + 5^e Y (A-o| 

l hcd idd J 


Y E E ^F r fif( D ^+ e j )g( D ^+ e i) 

neN J 0 [j&Jo\Diej 0 \D 

E E n(D,n)f(D,n)g(D,n) l \+ Y IE( n j 


DC{1,...,J} ng f^ 


j£j\D 


For (/, Q z 'g)j r the last two lines in the respective formula are identical to those in the 
displayed formula. The difference therefore is (/, Q z g)n ~ (/, Q Z g)n = 


Y Y ’K- 0 ) IE ^ Y ( D ^ H )f( D ^)g( H ,n)+Y ( i Y ( D ^ I )f( D ^)g( I ,!i) 

lg PJ J DC{1,...,J} {HcD IdD 

- Y E ^( D )f( D ^)9{D,n) | E (1 Y ( D , H ) + E 9 ^- 0 ’ 7 )} 

DC{1,...,J} l HCD IDD J 

- Y E i E qY ( D ^ H )f( D ^)g(H,n) + ^g Y '(D,/)/(T),n)^(/,n)| 

„ g ^ J DC{1,...,J} {HCD IDD J 

Y *(&) E ^i. D )f^ D ^)g{D,n) | E q Y '(D,H) + ^g Y, (D,/)| 

pjj dc{i,..,j} l//cf /of J 


neN 

For fixed n we interpret f(D,n) and g(D,n) as functions of D parametrized by X t (uj ) = n. 
This leads to 

(f,Q Z g)n- {f,Q Z 'g)ir= Y (f(°^),Q Y g(-,n)(o)}^-(f(o,n),Q Y 'g(-,n)(o)} j 

neN J 


= E m 


(/(V), Q Y g(-, X,)(o)), - {/(O, Xt), Q Y 'g (-, X t )(o)) f 


(3.3) 


ii 











□ 

Theorems 18.31 and 13.41 state a reduction of complexity: Comparing operators and resulting 
one step correlations via functions on infinite space is reduced to comparing matrix operators 
via functions on finite space. The theorems are valid for all square integrable functions /, g 
on E. This opens the way to compare correlations for multidimensional marginals in time 
of the network processes with unreliable nodes according to concordance ordering utilizing 
the abstract setting for Markov processes of |DS08j [Theorem 5.2], 

For a concise notation we introduce the standard difference operators for functions on N J . 
For all / e L 2 and all j — 0,1,..., J, we define (recall eo is the zero vector) 


Vjf : E 


(D,n) -)• V j f(D,n) := f(D,n + e,) - f(D,n), 


and 

Vf : E —► M J+1 " |D| , (D,n) (V,f(D,n),j E J 0 \ D). 

That way we can treat Df(D,n ) as a vector of the dimension corresponding to the size of 
D, and the corresponding routing matrices R D as operators on it. Moreover, it is possible to 
consider the corresponding scalar products generated by invariant vectors £ D , and write the 
formula for the difference of one step correlations (which are scalar products with respect 
to invariant measures for the network process n) in terms of scalar products with respect to 
invariant measures £ D for the routing processes. 


Corollary 3.5 For unreliable Jackson network processes Z = (Y, X) with £ D the probability 
solution of the equation x D = x D ■ R D and = J2jeJ\D ^e total service rate for 

nodes which are up, we have 


(f,Q z g)* 


Ejr 


A(( V + Id)f(Y t , X t ), R y \V + Id)g(Y t , X t )) Yt 
so 


En [(f(o,X t ),Q Y g(-,X t )(o))t] 


E^ 


f(Y t , X t )g(Y t , X t )(\ + p Y J 


Proof. Take Proposition 13.21 (13.2(1 , condition as in Theorem 13.31 and insert the suitable 
difference operators. □ 

We can now reformulate the result of Theorern l3.3l in a more compact form which immediately 
relates our results to methods dealt with in optimizing MCMC simulation. 


Corollary 3.6 For unreliable Jackson network processes Z, Z' as in Theorem \‘3.‘J\ we have 


if, Q z g)n - {f, Q z 'g)n = E * 


f Yt 
Lso 


(V + Id)f(Y u X t ), (R y - R ! Yt ) (V + Id)g(Y t , X t ) 




4 Applications 

The proofs of the applications below follow the ideas which are used to prove the theorems 
in Section 13.21 We therefore give only hints to perform the proofs here. 
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4.1 Comparison of spectral gaps 

Let Z be a continuous time homogeneous ergodic Markov process with stationary probability 
7 f and generator Q z . Let 7 r(/) = J^f(x)7t(dx) The spectral gap of Z, resp. Q z is 

Gap(Q z ) = inf{(/,-Q z />* : / e L 2 (E, t), tt(/) = 0, </,/)* = 1}. 

The spectral gap determines for Z the speed of convergence to equilibrium 7 f in L 2 (E, tt) 
with norm || ■ ||*.: Gap(Q z ) is the largest number A such that for the transition semigroup 
P = (P t : t > 0) of Z holds 

l|f</ - #(/)ll« < e- A ‘||/ - #(/)||» V/ 6 L 2 (E,S). 

It should be noted that one has to be careful which class of functions is used for the definition 
of spectral gap. For a discussion and more references see the introduction of [ LS14 ]. 

We utilize the following orderings to compare routings, failure processes and then correla¬ 
tions, see [Pes 73]. 

Definition 4.1 Let R = [r^] and R' = [rh] be transition matrices on a finite set E such 
that £R — £R' — £ for a probability vector £. 

R' is smaller than R in the positive semidefinite order, R’ -< p d R, if the matrix R — R' is 
positive semidefinite. 

R' is smaller than R in the Peskun order, R' -<p R, if for all j,i 6 E with i j holds 

r'ji < Tji. 

Peskun used the latter order to compare reversible transition matrices with the same station¬ 
ary distribution and their asymptotic variance, and Tierney |Tie98] has shown (in a more 
general setting, i.e. using operators rather than matrices) that the main property used in 
the proof of Peskun, namely that ” R -<p R' implies R' -< p d R”, holds without reversibility 
assumptions. 

Example 4.2 For any transition matrix R = [r^] and R' = Id (of the same dimension as 
R) holds R' -<p R , so the family of transition matrices of fixed dimension has a (unique) 
minimal element. Therefore STALLING from Example 12.51 is an extremal (re-)routing scheme, 
because it utilizes Id in case of any breakdown. 


Proposition 4.3 Consider two ergodic unreliable Jackson networks with state processes Z 
and 7J and with the same arrival and service intensities, and the same failure-repair rates. 
Assume that the equations x — x ■ R and x — x ■ R' have the same normalized solution £, and 
the Assumption ^. 21 holds, i.e. both networks follow some rerouting mechanism according to 
(12.8)) with the property (I2.9[) . 

If R D -< p d R' D for all D (also for D = %), then 

Gap{Q z,) < Gap(Q z ). 


Proof. From Corollary 13.61 we have for all / 6 L 2 


(f,-Q z fh-(f,-Q z 'f)« 




A((V + Id)f(,Y„X t ),(R' Y ‘ 


R y ‘)(V + Id)l(Y t ,X t )) iY> 
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and from the product formula we rewrite this formula as 


{f, -Q z fh-(f,-Q z 'fh = E n Et 


A 


Ui 


Y t 


{{V + Id)f(Y t , X t ), (R ,Yt - R Yt ) (V + Id)f(Y t , X t )) 






D 


ii 


D 


(V + Id)f(D, X t ), ( R ,d - R d ) (V + Id)f(D, X t )) 


From -< p d ordering of routings for all / G L 2 therefore (/, —Q z f)n > (/, —Q z 'f)i holds. □ 


Remark 4.4 If R -< p d R' and the rerouting for Z! is by stalling, then by the extremal 
property of Id for -<p the assumptions of Proposition \4-3[ are fulfilled. 


Computing spectral gaps for Markov processes with multidimensional state space is chal¬ 
lenging, in many cases nearly impossible. Exceptions are multidimensional independent 
birth-death processes, because for birth-death processes explicit results are known, see e.g. 

Doo 02 j , and Liggett has proved that the gap of independent processes is the minimum of the 
spectral gap of the marginal processes [Lig89| [Theorem 6.2], We will show that the gap of 
the joint queue length network process Z (with unreliable nodes) can be bounded from below 
by the gap of a network process consisting of identical breakdown-repair process and related 
multidimensional birth-death process with conditionaly independent components. 


Proposition 4.5 Consider an ergodic Jackson network process Z with unreliable servers as 
in Theorem m Assume that for all D C {1, 2,..., J} with tt(D) > 0 the routing matrix 
R D = jG j 0 \rj has strict positive entrance and departure probabilities (rf\ > 0,r^ > 0) 
for every node i 6 J\ D. 

Assume further that for all D C {1, 2,..., J} the rerouting R D fulfills overall balance for all 
network nodes which are up, i.e. 

J? r P = e J\ D - (4- 1 ) 

idJ\D ieJ\D 


Then there exists an ergodic Jackson network process process Z' with unreliable servers as 
in Theorem\2. 7| with the same stationary distribution it as Z, such that 


Gap{Q z ') < Gap{Q z ). 


The nodes of ZI are perturbed by a common breakdown-repair regime identical to that of Z, 
and for any given set D of broken down nodes the joint network process on J\D consists of 
conditionally independent birth-death processes, and the coordinate birth and death processes 
on the i-th coordinate have birth rate A r^ and state dependent death rate pi(n iK- 


Proof. In order to obtain a lower bound for the spectral gap using birth and death processes 
the idea is to allow in the comparison network for each node i, which is up, that any customer 
who enters node i from the external source after being served only to feed back (possibly 
iteratively) to node i or to depart from the network. This results in updating the service 
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rates suitably. Recall that the original network processes and the constructed comparison 
processes are additionally perturbed by the same failure mechanism. 

Consider the situation when nodes in D are down. Directly from the generator of the 
network process Z in (I2.10j) . it is clear that after reducing movements inside the network, 
and allowing only for movements into the network from outside, or from the network into 
outside, or feedback, we get as long as the reliability level D does not change transitions for 
changing the queue lengths which look identical as those of the generator of independent 
birth and death processes such that on the z-th coordinate the birth rate equals Aand the 
state dependent death rate equals /i,;(n,;)r ) ((, i G J\D. 

Now, in order to be able to apply a formula for differences of one step correlations we have 
to show on every reliability level D that such a modification is possible within a class of 
networks with extended routings having the same stationary solution. For this reason we 
need the assumption on overall balance (14.ip . 

More precisely, we define R' D by r'Jff = rff , rff = rff, for all z, r'T = 0 for j z, i,j G J\D, 
and r[f = 1 — rff for z G J \ D. With the routing R ,D , the network process Z' (when nodes 
in D are down) develops as a vector of independent birth and death processes for the up 
nodes in J\D. 

For j G {1,..., J} let rfR be the solution of the traffic equations for R' n . We have directly 


riT = Arg + rfrg, j G J \ D , (4.2) 

and the solution of this system is uniquely defined. We show that (14.2j) is solved (for each 
D) by (r}f,j G J \ D) as well. Inserting this into (14.21) we obtain with r'R = 1 — r® 0 for 
jeJ\D 


V? = Arg + rjf (1 - 


n 


jo) = K + ’( 


ieJ\D 


— A r D 

- Ar 0j 


E 

ieJ\D 


n D r D 

'll IJ 


which is the traffic equation when nodes in D are down in Z and has the unique solution 
£ J \ D). The last step is: First observe that (r/j 5 , j e J \ D) is by Assumption 12.21 
the restriction of ( rjj,j G J ) (the solution of the traffic equation with all nodes in Z up), 
to J \D, and, secondly, consider the above constructed system of independent birth-death 
processes for the reliability level 0 as the comparison system when all nodes are up with 
routing R', and the R ,D as rerouting scheme for this network on reliability level D. 

Because rjj = rf,- and rj® = r)'P for all j G J\D and all D, the 7?' D are obtained by restricting 
(■ rjj : j G J) (the solution of the traffic equation with all nodes in Z' up), to J\D. 

Note that for all D holds R ,D -<p R D , therefore R D -< p d R ' D , and the result follows from 
Proposition 14.31 □ 


Remark 4.6 The overall balance (14.11) of the rerouting R D for all network nodes which are 
up is satisfied whenever R is a reversible transition matrix. 

If the overall balance (14.11) holds for J only (i.e., for D = (I)), and the rerouting for Z' is by 
stalling, the extremal property of Id for -< P leads to the same conclusion as the assumptions 
of Proposition |R 
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Remark: From irreducibility of Z, for i,j e J,i j we obtain from 04.11) = r/jr'^ 

but this does not mean that the matrix i? is reversible, because r^ 1 ’^ r^ may hold. 


The lower bound Gap(Z') in the previous statements is of interest, because it has con¬ 
stitutive processes with conditionally independent coordinates. From Lig89] [Theorem 2.6] 
it is known, that the gap of a process with independent coordinates is the minimum of 
the gaps of the coordinate processes. Unfortunately enough, this theorem does not apply 
here directly, because the coordinate birth-death processes are controlled by the common 
breakdown-repair process. However, the comparison result of Proposition 14.31 can be used 
to obtain upper bounds for spectral gaps. This topic will be considered in a separate paper. 
Nevertheless the bound is of practical value, because the bounding process 7J is reversible 
with respect to n, which can be seen by checking the local balance equations. As a con¬ 
sequence, the bounding techniques for reversible processes, e.g., using Cheeger constants, 
found in the literature can be applied directly. 

In |LSl4 j [Example 6.2] it is shown, that the bounds obtained via Proposition 14.51 can be very 
good for networks with reliable nodes. They compare the bound for an example provided in 


|TRT12j : 

This is the network described in Section l2Tl with state independent service rates p,j = /x, and 
routing matrix which fulfills ry — 0, Vi = 0,1,..., J, and r 0i > 0, \/i = 1,..., J. Furthermore, 
for all i,j = 1,..., J, i j, holds complete symmetry by ry,- = p G (0,1 )/{j — 1), which 
results in r i0 = 1 — p( J — 1) > 0, Vi = 1,..., J. 

It is assumed that no breakdowns (and repair) occur. 

In this symmetric network the partial balance (for D = 0) holds if and only if ro* = 1/J, Vi = 
1,..., J, which implies X 3 = A/ J, Vj = 1,..., J and rjj = A/(J(1— p(J — 1))), Vj = 1,..., J. 
We denote by p io := min{/Xj : 1 < j < J}, and, recalling the bound of the spectral gap 
obtained for birth-death processes by van Doom [ Doo02| . we obtain from the companion 
result of Proposition 14.51 (see I )S0S| [Proposition 4.4]) 


Gap{71) > 





It is easy to check, that for this setting the Assumptions (3.10) and (3.11) of Corollary 3.4 
in HIT 12 1 are fulfilled, which results in an upper bound for L 2 spectral gap 

GapiZ) < i - 2 ) ■ 

i-l+f-o) tends monotonously to J for p —> 1/(J — 1), while forp —)■ 0 it decreases monotonously 
to 1. 

From the ordering implication Peskun yields positive definiteness follows, that if we perturb 
routing of customers in the networks by shifting transition probability mass from non di¬ 
agonal entries into the diagonal (leaving the routing equilibrium fixed) then the speed of 
convergence of the perturbed process is smaller. 

The existence of L 2 spectral gap (that is the question when Gap(Q z ) > 0) for unreliable 
networks is a related topic. It is a common knowledge that for networks with constant ser¬ 
vice rates (not depending on the number of customers at node) the spectral gap for classical 
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Jackson network exists. For service rates that can depend on the number of customers the 
problem is more delicate. An iff characterization in terms of properties of service rates is 
given in Lorek and Szekli [ LS14 j. A special feature of such processes is that the existence of 
L 2 spectral gap is directly related to the tail properties of the stationary distribution. For 
references and details see mm- 

An analogue of Peskun ordering and positive semidehnite order for generator matrices is as 
follows. 

Definition 4.7 Let Q = ( q(x,y ) : x, y E E) and Q' = ( q'(x,y ) : x,y E E) be generator 
matrices on a finite set E such that fcQ = kQ’ = 0 holds for a probability vector i f. 

Q' is smaller than Q in the positive semidefinite order for generators, Q' -< p d Q, if Q — Q' 
is positive semidefinite. 

Q' is smaller than Q in the Peskun order for generators, Q' -<p Q, if for all x, y E E with 
x f- y holds q'(x,y ) < q(x,y). 

Lemma 4.8 Let Q = ( q(x,y ) : x,y E E) and Q' = ( q'(x,y ) : x,y E E) be generator ma¬ 
trices on a finite set E such that IrQ = nQ' = 0 holds for a probability vector tt. Then 
Q ~<p Q' =>■ Q' ~< P d Q holds. Proof. From q(x,y ) < q'(x,y) Vx,y E E with x y follows 
for all x E E that q’(x,x) < q(x,x) holds. So Q' — Q := ( q'(x,y ) — q(x,y ) : x,y E E) is a 
generator matrix as well. Therefore ~(Q' — Q ) is positive semidefinite. □ 


A direct consequence of Definition 14.71 this lemma, and of Theorem 13.41 follows from . 

Corollary 4.9 Suppose Z = (Y, X) and Z' = (Y 7 , X') are ergodic unreliable Jackson net¬ 
work processes, having the same arrival and service intensities, and with the same routing 
regimes, described by R = [?'i i j]i,j=o,i,...,j an d rerouting fulfilling Assumytion VJJA 
The breakdown-repair process for Z is given in Definition \2. 1\ and for Z' is defined similarly 
via functions Af B' : 2 J —> [0, oo), as given in Theorem \3.f\ 

The processes Y and Y' are Markov with generators Q Y = ( q Y (K, L) : K , L C J) of Y and 
Q Y ' = ( q Y '(K,L ) : K,L C J) of Y' as defined in (j2.5jl for Y and similar for Y'. 

Assume that the stationary distributions of Y and Y' are identical, denoted by if. 

If Q Y -< p d Q Y ' holds, then Gap(Q Y ') < Gap(Q Y ). 

An easy to understand property is that whenever the breakdown-repair process Y of Z 
is uniformly faster than the breakdown-repair process Y' of Z', i.e., for all x j- y holds 
q Y ( x,y ) < q Y (x,y), we have Gap(Q Y ) < Gap(Q Y ). This follows directly from Lemma 14.81 
So, for example, if we have A(D) = ■ A\D ), and B(D ) = ifi D \ ■ B\D ), A E for some 

k > 1, then Q Y -<p Q Y , and these breakdown-repair processes fulfill the requirement of 
Corollary 14.91 

4.2 Asymptotic variance 

Peskun (Pes73j and Tierney |Tie98j derived comparison theorems for the asymptotic variance 
of Markov chains for application to optimal selection of MCMC transition kernels in discrete 
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time. These asymptotic variances occur as variance in the limiting distribution of central 
limit theorems (CLTs) for the MCMC estimators. For our network processes Z we consider 
Markov chain (Xk, k > 1), say with transition matrix K = Id + eQ z (with e > 0 sufficiently 
small). Under some regularity conditions on a homogeneous Markov chain with one step 
transition kernel K we can obtain CLT of the form 

1 n 

/(**) - £*(/(*,))) ® JV(0,«(/, A-)), 

1 k= 1 

where the asymptotic variance is w(/, K) = (/, /)* — 7ir(/) + 2 ^fcli(/) K k f)*- 
Regularity conditions under which CLT holds for such Markov chains is a topic which we 
shall study in a separate paper. For reversible chains with positive spectral gap it is possible 
to give conditions in terms of the service rates, but a general iff characterization in terms of 
the service rates seems to be an open problem. 


Proposition 4.10 Consider two ergodic unreliable Jackson networks with the same arrival 
and service intensities, and state processes Z and 7J . Assume that the routing matrices R 
and R' are reversible with respect to f. Both networks follow a rerouting mechanism according 
to ( 12 . 8 |) with the property ( 12 . 91 ) , such that R D and R' D are reversible with respect to f D . 

If R D and R' D are ordered for all D in positive definite order, R' D -< pd R D , then for any 
function f E Lq(E, 7t) := {g E L 2 ( E, ff) : n(g) = 0} holds v(f, Id + eQ z ) > v(f, Id + eQ z '\ 


Proof. For standard Jackson networks without breakdown and repair it is well known that 
reversibility of the routing matrix R implies reversibility of the joint queue length process. A 
direct way to prove this is to check the local balance equations with respect to the stationary 
distribution tt. It is easy to see that this way of proof verifies reversibility of the processes 
Z and Z' here as well. The reason is that the breakdown and repair process is reversible 
with respect to 7r, and that for fixed D and No intensities of possible transitions on 


balance locally with respect to the densities IkeJ\.D YYi=i yjjjj 
Because Z and Z' are irreducible we can apply a result of Mira and Geyer [MG99j [Theorem 
4.2], which states that under this condition the required inequality is equivalent to ordering 
of the one step correlations for / E Lg(E, fr). The letter statement can be shown exactly as 
in the proof of Proposition 14.31 □ 


5 Jackson networks with finite waiting rooms 

We consider the Jackson networks from Section l2Tl and assume now that some or all of the 
single server nodes with queue length dependent service intensities have a finite capacity. 
This means that such nodes have a service place and only a finite number of waiting places 
available to store waiting customers. The problem one is faced with is that whenever a node, 
say j, reaches its maximal buffer size then no further arriving customer can enter node j 
and we have to ’’resolve” such blocking situation. 

There are many ’’blocking protocols” available in practice and in theoretical models but the 
standard observation is that when reducing the infinite buffers of the Jackson networks from 
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Section [2TT1 to finite sizes the stationary distribution is no longer of product form, in fact in 
almost all cases the stationary distribution is not available. The simplest reaction to a full 
buffer situation with a newly arriving customer is to discard this customer - this leads to a 
simple equilibrium only in case of a single station network. 

A survey, emphasizing applications in telecommunication systems is given in [ Per90] [Section 
2 ], a general overview is I 1 11 )( )()! . 

It turned out that for obtaining product form stationary distributions for finite buffer Jackson 
networks one has to introduce clever rerouting strategies, i.e. to redirect and redistribute 
newly arriving customers who want to enter nodes with full buffers in such a way that the 
the system’s customer flow is balanced. A literature review shows that the most prominent 
rerouting strategies in case of full buffers which lead to product form equilibrium are the 
schemes mentioned in Example 12.51 These are described in more detail e.g. in |Dijll| [Section 
1.6.3] under the headings Conservative protocol = Stalling, Jump-over protocol 
= Skipping, and in [PerQOj [p. 456,457] as repetitive-service-random destination. It 
turns out that in all three cases the stationary distribution for feasible states in the hnite 
buffer network are (up to normalization) of the form (12.4(1 . A detailed study of the jump-over 
protocol in case of blocking is |Dij88| . 

As a referee pointed out, it is therefore a tempting conjecture that correlation formulas 
similar to ours should be valid in the context of hnite buffer networks as well. But going 
into the details shows that there are subtleties in the structure and the functioning of the 
hnite buffer networks which must be handled careful. 


(1) The network of Section [2.41 is influenced by an external environment which is Markov 
for its own, and changes of the environment enforces the Jackson network to redirect its 
customers, if necessary. In case of the hnite buffer network the customers are redirected by 
intrinsic forces (state space restrictions). 

(2) The intrinsic forces which influence routing and rerouting lead to queue length depen¬ 
dent routing decisions. Seemingly, this makes it nearly impossible to formulate an overall 
simple condition in parallel to Assumption 12.21 which produces product form stationary dis¬ 
tributions. 

(3) The area of applications (problems) which is the classical source of interest in hnite 
buffer networks with blocking protocols is in telecommunications networks. There it is un¬ 
realistic to assume that in case of blocking (=full buffer) the service at the full buffer node 
is interrupted, which is natural at broken down nodes in the setting of Section 12.41 

Nevertheless, it turned out that correlation formulas similar to those in Propositions 13.21 and 
13. 1 1 can be derived in case of the most popular unblocking scheme (repetitive-service-random 
destination) for networks with reversible routing. This and more general results are part of 
our ongoing research and will be summarized in a forthcoming technical report [DS15] . 
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6 Proof of the correlation formulas 


For /, g : E —> M. and steady state probability fc of Z we are interested in the one-step 
correlation expressions 


J n t 


{f,Q z s)» = C - 1 £ E^>nn(^i /(A«)l £ \?g{D,n + . 

DC{l,...,J} ReN J <=!»=! [jeJ\D 

+ Vj('n j ) r ? 0 g(D,n-e j )+ ^ fi j (n j )rf i g(D, n — ej + e;) 


( 6 . 1 ) 


jeJ\D 


j£J\D i£j\D,i^j 


X f + ^K)( 1 _r S) ) 9 (D,n) 

\j£j\D j£j\D 


+ ^ q Y (D,H)g(H,n) + ^ q Y (D, /)<?(/, n) - ( ]T q Y (D,H ) + ^g Y (D,J) ) <?(£>, n)} 

HcD IDD \HcD IdD ) J 

Proof, (of Proposition 13.ip Interchanging summations, regrouping terms, and exploiting the 
product form structure of the state distribution, we obtain from (16.ip 


711 fe) 

neN J 


q Y ( D ’ H )f( D ’™)9{ H ,R) + ^2q Y ( D ,i)f{ D ,!i)g{i,ii) 


.HcD 


IDD 


-l(D,n)g(D, n) \ £ q Y (D, H) + £ q Y (D, I) 


„HcD 


IDD 


n e 


e e nK'nj) 

|o| l^D \ i =1 ' . 




,eN'' 


S n ( c t x f 9< < D ^ + e j ) 

„ AD eN J “ |D| J\d V <=i e J [ieJ\D 

+ 9 j( n j) r fo9(D,(n D ,n JxD -e j ))+ ^ ^{n^rf^D, (n D ,nj XD - e, + e*)) 

jeJ\D jeJ\DieJ\D,i^j 

X f + S ^K)( 1_r S)J d(D, (n D , nj\ D )) 
jeJ\D jeJ\D J 

For each fixed D,n D , the terms in the last squared brackets are identical to the one step 
correlation of a Jackson network in equilibrium on node set J\D (with the respective tran¬ 
sition rates) with respect to the functions f(D , (n D , (-)j\ D )) and g(D, (n D , (-)j\d))- 

We have agreed to denote the generator of such network by Q J \ D , and its steady state by 
n j\Di which leads to the proposed formula with the aid of the synthetic networks. □ 
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Proof, (of Proposition I3.2[) We restart with the expression (16.11) and observe that for fixed 
D C {1 ,..., J} the contribution of —rP in the negative terms would be exactly the contri¬ 
bution in the double sum of i £ J\ D,i — j in the positive terms, where for i — j would 
occur g(D,n — ej + ej) = g(D,n) otherwise. Together with 0) = 0 Vj, incorporating these 
contributions simplifies our expression to 


J ri£ 


-c-' E E*u»im Wj) 




f (D, n)g(D, n) 


(6. 


(e q y (d,h)+j2q y ( d , 1 )] + (E A ? + E /pK 

\HcD IdD J \jeJ\D j£J\D 


J n e 


+c-' e e ^>nn 


neN J 


Ve 


f(D,n)\ Y, 1 Y (D,H)g(H,n) + Y,‘! t (DJ)g(I,n) 


, HcD 


IDD 


J n e 


+C E E E Afs(C,a+ei) 

neN J l=li=l \^\)J ( je J\ D 


E f J >i( n 3) r ?o9(D,&-e j )+ E E ^A n j) r fi9{ D ,R ~ + e*) 

j&J\D j&J\D i&J\D 


The last two lines of the formula above turn into 

ne , 

c-' E *<*> e nn(jy 

„„ e H DfeDi= i 

Vi 


lld 

nf> 
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In the last line, if rij >0, the expression Hj{nf) cancels against a factor in the steady state 
probability. This leads to (we underbrace some intuition and use rjj = rjP for j G J\D) 
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From Lemma [2.31 with t)q A the vector fj D := (r)®,j E Jo \ D) solves x = x ■ R D . Insert 
the normalized solution := (f[ J : i G Jq\D) into (16.31) . and then insert into the correlation 
expressions (j6.2[) to obtain 
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With EjeAt> J + A Tq 0 = A this yields finally the desired correlation formula from Proposi¬ 
tion |3]2j □ 
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